{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Implementation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This section will demonstrate how to fit the discriminative classifiers discussed in this chapter with `scikit-learn`. Note that other libraries are frequently used—e.g. `statsmodels` for logistic regresssion and `tensorflow` for the perceptron. \n",
    "\n",
    "For binary tasks, we'll be using the {doc}`breast cancer </content/appendix/data>` dataset and for multiclass tasks, we'll be using the {doc}`wine </content/appendix/data>` dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "from sklearn import datasets\n",
    "\n",
    "# import data\n",
    "cancer = datasets.load_breast_cancer()\n",
    "X_cancer = cancer['data']\n",
    "y_cancer = cancer['target']\n",
    "wine = datasets.load_wine()\n",
    "X_wine = wine['data']\n",
    "y_wine = wine['target']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Logistic Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Binary Logistic Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A standard `scikit-learn` implementation of binary logistic regression is shown below. Note the two arguments set when instantiating the model: `C` is a regularization term where a higher `C` indicates *less* penalty on the magnitude of the coefficients and `max_iter` determines the maximum number of iterations the solver will use. We set `C` to be arbitrarily high such that there is effectively no regulariation and `max_iter` to be 1,000, which is enough for this model to converge. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=100000, class_weight=None, dual=False, fit_intercept=True,\n",
       "                   intercept_scaling=1, l1_ratio=None, max_iter=100000.0,\n",
       "                   multi_class='auto', n_jobs=None, penalty='l2',\n",
       "                   random_state=None, solver='lbfgs', tol=0.0001, verbose=0,\n",
       "                   warm_start=False)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "binary_model = LogisticRegression(C = 10**5, max_iter = 1e5)\n",
    "binary_model.fit(X_cancer, y_cancer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`scikit-learn`'s logistic regression model can return two forms of predictions: the predicted classes or the predicted probabilities. The `.predict()` method predicts an observation for each class while `.predict_proba()` gives the probability for all classes included in the training set (in this case, just 0 and 1)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training accuracy: 0.984182776801406\n"
     ]
    }
   ],
   "source": [
    "y_hats = binary_model.predict(X_cancer)\n",
    "p_hats = binary_model.predict_proba(X_cancer)\n",
    "print(f'Training accuracy: {binary_model.score(X_cancer, y_cancer)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multiclass Logistic Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Multiclass logistic regression can be fit in `scikit-learn` as below. In fact, no arguments need to be changed in order to fit a multiclass model versus a binary one. However, the implementation below adds one new argument. Setting `multiclass` equal to 'multinomial' tells the model explicitly to follow the algorithm introduced in the {doc}`concept section </content/c3/s2/logistic_regression>`. This will be done by default for non-binary problems unless the `solver` is set to 'liblinear'. In that case, it will fit a \"[one-versus-rest](https://www.google.com/search?q=one+versus+rest+classifier&oq=one+versus+rest+classifier&aqs=chrome..69i57j0l4j69i64.3569j0j1&sourceid=chrome&ie=UTF-8)\" model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=100000, class_weight=None, dual=False, fit_intercept=True,\n",
       "                   intercept_scaling=1, l1_ratio=None, max_iter=10000,\n",
       "                   multi_class='multinomial', n_jobs=None, penalty='l2',\n",
       "                   random_state=None, solver='lbfgs', tol=0.0001, verbose=0,\n",
       "                   warm_start=False)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "multiclass_model = LogisticRegression(multi_class = 'multinomial', C = 10**5, max_iter = 10**4)\n",
    "multiclass_model.fit(X_wine, y_wine)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Again, we can see the predicted classes and predicted probabilities for each class, as below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training accuracy: 1.0\n"
     ]
    }
   ],
   "source": [
    "y_hats = multiclass_model.predict(X_wine)\n",
    "p_hats = multiclass_model.predict_proba(X_wine)\n",
    "print(f'Training accuracy: {multiclass_model.score(X_wine, y_wine)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Perceptron Algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The perceptron algorithm is implemented below. This algorithm is rarely used in practice but serves as an important part of neural networks, the topic of Chapter 7. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import Perceptron\n",
    "perceptron = Perceptron()\n",
    "perceptron.fit(X_cancer, y_cancer);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fisher's Linear Discriminant "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we fit Fisher's Linear Discriminant with the `LinearDiscriminantAnalysis` class from `scikit-learn`. This class can also be viewed as a generative model, which is discussed in the next chapter, but the implementation below reduces to the discriminative classifier derived in the concept section. Specifying `n_components = 1` tells the model to reduce the data to one dimension. This is the equivalent of generating the \n",
    "\n",
    "$$\n",
    "f(\\bx_n) = \\bbeta^\\top \\bx_n\n",
    "$$\n",
    "\n",
    "transformations that we saw in the concept section. We can then see if the two classes are separated by checking that either 1) $f(\\bx_n) < f(\\bx_m)$ for all $n$ in class 0 and $m$ in class 1 or 2) $f(\\bx_n) > f(\\bx_m)$ for all $n$ in class 0 and $m$ in class 1. Equivalently, we can see that the two classes are not separated in the histogram below. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Separated: False\n"
     ]
    }
   ],
   "source": [
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n",
    "lda = LinearDiscriminantAnalysis(n_components = 1)\n",
    "lda.fit(X_cancer, y_cancer);\n",
    "\n",
    "f0 = np.dot(X_cancer, lda.coef_[0])[y_cancer == 0]\n",
    "f1 = np.dot(X_cancer, lda.coef_[0])[y_cancer == 1]\n",
    "print('Separated:', (min(f0) > max(f1)) | (max(f0) < min(f1)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAFYCAYAAAAGIGjZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de5xdZX3v8c+PEBgKWAIECISQcC0QTAgRsZaIUTTFcxAKVGiLQZF4A0urpyqelgi20Kqg7fEWBAGrclNAraKUclMUiZhAAAXEqAnhkshFCoQQfuePtUKHyZ6ZnZm9Zz8z83m/Xvs1s9d61lq/vbIz3/2s9ey1IjORJKk0G3W6AEmSGjGgJElFMqAkSUUyoCRJRTKgJElFMqAkSUUyoCRJRTKgJElFMqBGiIg4ISIyInZvMG/jet78btPmR0TT39KOiCMi4m9bVO6IFRFvj4j7IuK5iHi8ifY7RcQ3IuLRiHghIs7ppd2/RcS3Wlzr30TEHRHR79+Bde+XiNi4lTX0sq1XRcRlEfFgvR9XRcS1ETE3IsZ0r6fdtaizDKjR64vAqzag/RGAAdWHiNgRWADcAswGXt/EYhcCewFzgT8G/qXBencD3gl8tFW11j4PbFdvuwgRcSrwQ2Br4INU+/DtwL3A54D/1bnqNNTa/mlIZcrMZcCyTtfRrIjYNDNXd7qOfuwBjAEuyswf9Nc4InYAXgeckJnf6aPpqcDizFzYmjIrmflMRFwMfAD4UivXPRARMQs4B/h/mfm+HrOvrnuXmw99ZeoUe1CjVM9DJBGxZ0RcGRGPRMSzEfGbiLi8Pjx4IdWn7J3qwzwZEUt7rG9ORPwoIp6JiCci4qqI2KtHm+Mi4uf1+u+MiMMj4oaIuKFRbRExNSK+FxFPAZdFxO4R8eWI+FW9nQci4nMRMa6X5f+oXv6/69fztnr+8XUdT0XE9XUPpZl91utrrPfRutdxXb39C/tY1zeAFUAAF9Xtz2jQblPgr4Cv9pi+e0SsiYiP9pj+uYj4fUTMbOY1AZcA+0TEHzfZfu96nz0dESsi4ox1hwgj4uj6dUxr8DpuiIgf9bPuDwG/A/6u0czM/GVm3tHbws28P/p6n29IGw0Nd/jIM6bBf6QxTSz3beBx4N3ASmAn4DCqDzFnAuOBVwCH1+1f7M1ExBzgP4D/At4CbAGcAfwgIqZn5vKIOBT4CvBN4P3AtsCngC6qwzeNXA2cD/wz8AKwI1Wv71TgMWBX4DTgOzQ+XHk5cB7wCeA9wAURsQdwCNUfw7HAp6n++L+yr53T32us99FPgX8F3gvcDjzaxyr/HlgFHAscWk97oEG7g4CtgJu7T8zM+yPii8DfRMS/ZebKiPgHqsNhb9qA3tYi4ElgDtWhyf5cBVwAnAW8sX4dLwDz63kPUh2OfM+6BeoQfw3wtt5WWp9bOgS4KjOfbbL2npp5f/T1PmcD2mgoZKaPEfAATgCyn8f8bu3nV//8CVVYJHB4H+u/EFjWy7yFwH3Axt2mTQHWAOfUz28BlgDRrc2Mers39Fjf/Hr6X/fzmjcG/qRuu3+D5d/abdo44HmqUHhZt+nvq9vu0s+2mnmNr6/XdUiT/2aXAz/sp80HqQJgkwbzdgD+G/g4cCKwFvjzAbx3bga+30+bdfv0Qz2mnwf8HtiqW7sngM27tTmHKjA262P929frP6vJml98/zb7/mjyfd5vGx9D9/ATwchzJFVPp/vjoH6WWUX16f3siDip7mU0JSI2pwqaSzPz+XXTM/NXVCe7X1N/Op4JfD3rvwJ1m9uBX/Wx+it7bGuTiDitPjz3DFU4rOtZ7LXe0vDdbtt6DHgE+HFmPtmtzc/rnzsP5jX28Rr6Mh34WT9tdgSezMznes7IzIeoeqGnAF8A3peZlw2gjkfr7TSj5/ovoepNTq2fLwD+ADgOICK6qA4PX5yZzwygtqY18f5o5n0+4P8Laj0DauRZkpkLuz+oDj31qg6NQ6l6CWcB99bH79/dxPbGUZ1HWdFg3kNUo7G2pTqc9kiDNg/3se6e6zyL6pPzvwNvAg4E/qye19Vg+cd6PH+ul2m9Lb9OM69xg0TElsBu9B9QXXQ7nNrAfcCmwI8y8zMbWkftGWCzJtv2/Pda93wngMx8kOrQ7Lvq6cdQ7Z8v9LPeVXUduzRZRyN9vj+aeZ8P8v+CWsxzUAIgMx8A3hoRAUwDTgY+GxFLM/O7fSz6GNUhkR0azNuB6g/PSqpPs9s1aLM98Jveyurx/FiqT+IfWzchIrboo7ZWaeY1bqjpVKHXX0CtogrI9UTEbKo//D8CXh0R0zJzcbf5/0T14WBbqkNcy4E5mflUj1VtTfVv1Izteem5su3rn8u7Tfss1UCRA6jOR92cmXf3tdLMfL4eLHNoDHzEZr/vj2be54P4v6AWswell8jKIv7nO0/rDt2spsGn7Mz8b6oe2jH1oTwAImIXqu/13JiZa6k+kR5V/6df1+YAqvM4zfoDqqDrrtcT763SzGscwGr3p3otd/XT7ufA2IiY2H1iRMygGpTwRarBBb8B/qnHsjOACVSjAHelCtlZDbYxBfhFk3X/eY/nxwJPUZ1fBCAz/wu4h+rc06upvm/VjLOBbajOqa0nIqZExMv7WL7p90cf7/MNaqP2sgcl6v/0nwYuBe6nGvV3AtWggv+qm90NbF0f6lgIPJuZd9bz/p5qhNu3I+KzVOckPkp1svyTdZvTge8DV0bEAqpP9fOpDpG90GSp1wBzI+LOus4/owqIodDMa9wQ+wN3N9FTuKn+eSD199aiulrId6n25ymZ+UI93PyCiJiVmeuWmQG8JjOfrpfbhGoY94siYitgT6qRjs04qR5WfhvVKL53UA2+6XnVjM9TvadWAl9vZsWZeVNUVys5JyL2phqY8xuqHuTr6m39BdDbUPM+3x/NvM+b/L+godLpURo+WvPgf0bx7d5g3sb0PYpvO+AiquHeT1P9EbsReGO39psDX+N/Dnct7bGNOVSHmp6h+qN9NbBXjzZ/QfVJfTVVz+FIqkNcV/ZoN7/exsY9pm9LdVL+sfrxFapBIEn1Zdf+ll8K/HuPaYfUbV/fxD7u8zWyAaP4qIZ3X9jkv+2twJfq33egOsR2A7BptzZjqHott9TPJwGPdps/lqqns1mPdf8l8CywTT81rNunU4Hr633wENXw+o0atJ9Qt//4AN7Lf0w1wnEFVY/od1Rh/FfrtkWDUXz9vT9o7n3ebxsfQ/eI+h9FGnL1Yav7gX/MzDM7XU+pIuIEqk/1E7LuDTWxzBHAvMw8rH4+gyoQX96j3XeBlZl5fItrPonq/NiemXl/K9et0cNzUBoSEbFZ/a3+oyLiNVFd1eFaqk+pX+xweaX7MtUghPf017CbA6gOxa4zs8dz6i8Yv5YWXuMvIvaJiP9dr/Mqw0mDYQ9KQ6I+/3Ep1XeytqH6gunNwGmZuaSvZQURcRAwIzM/28J1zgHGZebXWrjOG6gO0d0C/EVWw86lATGgJElF8hCfJKlIBpQkqUhD+j2oOXPm5DXXXDOUm5QklS8aTRzSHtTKlc1eTUWSNNp5iE+SVCQDSpJUJANKklQkLxYrSS20Zs0ali1bxrPPDvTO9SNXV1cXEydOZOzYsU21N6AkqYWWLVvGlltuyeTJk+l2d5lRLzNZtWoVy5YtY8qU5u6y4yE+SWqhZ599lm222cZw6iEi2GabbTaoZ2lASVKLGU6Nbeh+MaAkaYR56KGHOPbYY9ltt93YZ599OOyww7j33ntZunQpU6e258bAq1ev5i1veQu77747r3zlK1m6dOmg1+k5KElqo5vuau1giVn7dvU5PzM58sgjmTt3LpdccgkAixYt4uGHH2bnnXduaS3dnX/++YwbN47777+fSy65hA9+8INceumlg1qnPShJGkGuv/56xo4dy7ve9a4Xp02fPp2DDz74Je2WLl3KwQcfzIwZM5gxYwa33HILACtWrGDWrFlMnz6dqVOncvPNN7N27VpOOOEEpk6dyn777ce555673navvvpq5s6dC8DRRx/Nddddx2DvlmEPSpJGkCVLlnDAAQf022677bbj2muvpauri/vuu4/jjjuOhQsX8tWvfpU3vvGNfOQjH2Ht2rU8/fTTLFq0iOXLl7NkSXXrtscff3y99S1fvvzFHtrGG2/MH/7hH7Jq1Sq23XbbAb8WA0rSiLVgweJe582bN20IKynPmjVrOPnkk1m0aBFjxozh3nvvBeAVr3gFb3/721mzZg1HHHEE06dPZ9ddd+WBBx7glFNO4U1vehNveMMb1ltfo97SYAeLeIhPkkaQfffdl5/+9Kf9tjv33HPZfvvtWbx4MQsXLuS5554DYNasWdx0003stNNOHH/88Vx88cWMGzeOxYsXc8ghh/CZz3yGd7zjHeutb+LEifz2t78F4Pnnn+eJJ55g6623HtRrMaAkaQSZPXs2q1ev5rzzzntx2m233caNN974knZPPPEEEyZMYKONNuLLX/4ya9euBeDXv/412223HSeddBInnngit99+OytXruSFF17gqKOO4swzz+T2229fb7uHH344F110EQBXXHEFs2fPHnQPykN8kjSCRARXXnklp556KmeffTZdXV1MnjyZT33qUy9p9573vIejjjqKyy+/nNe+9rVsvvnmANxwww18/OMfZ+zYsWyxxRZcfPHFLF++nLe97W288MILAJx11lnrbffEE0/k+OOPZ/fdd2frrbd+cQThoF7LYEdZbIiZM2fmwoULh2x7kka3TpyDuueee9h7773bsu6RoJf90/kbFkqS1CwDSpJUJANKklQkA0qSVCQDSpJUpH4DKiK6IuInEbE4Iu6KiI/W0y+MiF9FxKL6Mb395UqSRotmelCrgdmZOQ2YDsyJiIPqef8nM6fXj0Vtq1KS1LRO3G7jpptuYsaMGWy88cZcccUVLVlnv1/UzeqLUk/VT8fWj6H78pQkDWN9fRdrIPr7/lanbrcxadIkLrzwQj7xiU+0bJ1NnYOKiDERsQh4BLg2M2+tZ/1jRNwREedGxKYtq0qSNCCdut3G5MmTefnLX85GG7VuaENTlzrKzLXA9IjYCrgyIqYCHwYeAjYBFgAfBM7ouWxEzAPmQZWwkrQhvCL5hunU7TbaYYOiLjMfB24A5mTmiqysBr4EHNjLMgsyc2Zmzhw/fvygC5YkDd6aNWs46aST2G+//TjmmGO4++67gep2G1/60peYP38+d955J1tuueVLbrdxzTXX8LKXvWxIamxmFN/4uudERGwGvB74eURMqKcFcASwpJ2FSpL616nbbbRDMz2oCcD1EXEHcBvVOahvA1+JiDuBO4FtgY+1r0xJUjM6dbuNdmhmFN8dwP4Nps9uS0WSpAHr1O02brvtNo488kgee+wxvvWtb3H66adz1113De61eLsNSSUbzCAJb7dRHm+3IUka9gwoSVKRDChJUpEMKElqsaE8tz+cbOh+MaAkqYW6urpYtWqVIdVDZrJq1Sq6urqaXqapSx1JkpozceJEli1bxqOPPtrpUorT1dXFxIkTm25vQElSC40dO5YpU6Z0uowRwUN8kqQi2YOS1HGtvmeSRgZ7UJKkIhlQkqQiGVCSpCIZUJKkIhlQkqQiGVCSpCIZUJKkIhlQkqQiGVCSpCIZUJKkIhlQkqQiGVCSpCIZUJKkIhlQkqQiGVCSpCIZUJKkIvUbUBHRFRE/iYjFEXFXRHy0nj4lIm6NiPsi4tKI2KT95UqSRotmelCrgdmZOQ2YDsyJiIOAfwbOzcw9gMeAE9tXpiRptOk3oLLyVP10bP1IYDZwRT39IuCItlQoSRqVmjoHFRFjImIR8AhwLfBL4PHMfL5usgzYqT0lSpJGo6YCKjPXZuZ0YCJwILB3o2aNlo2IeRGxMCIWPvroowOvVJI0qmzQKL7MfBy4ATgI2CoiNq5nTQQe7GWZBZk5MzNnjh8/fjC1SpJGkWZG8Y2PiK3q3zcDXg/cA1wPHF03mwtc3a4iJUmjz8b9N2ECcFFEjKEKtMsy89sRcTdwSUR8DPgZcH4b65QkjTL9BlRm3gHs32D6A1TnoyRJajmvJCFJKpIBJUkqUjPnoCSpSAsWLO7IeufNm9aW7eql7EFJkopkQEmSimRASZKKZEBJkopkQEmSimRASZKKZEBJkopkQEmSimRASZKKZEBJkopkQEmSimRASZKKZEBJkork1cwljUrtuhK6WscelCSpSAaUJKlIBpQkqUgGlCSpSAaUJKlIBpQkqUgGlCSpSAaUJKlIBpQkqUj9BlRE7BwR10fEPRFxV0T8dT19fkQsj4hF9eOw9pcrSRotmrnU0fPA+zPz9ojYEvhpRFxbzzs3Mz/RvvIkSaNVvwGVmSuAFfXvv4+Ie4Cd2l2YJGl026CLxUbEZGB/4Fbg1cDJEfFWYCFVL+uxBsvMA+YBTJo0aZDlSlLZ+roI7bx504awkuGv6UESEbEF8HXg1Mx8EvgcsBswnaqH9clGy2XmgsycmZkzx48f34KSJUmjQVMBFRFjqcLpK5n5DYDMfDgz12bmC8B5wIHtK1OSNNo0M4ovgPOBezLznG7TJ3RrdiSwpPXlSZJGq2bOQb0aOB64MyIW1dNOA46LiOlAAkuBd7alQknSqNTMKL4fANFg1ndaX44kSRWvJCFJKtIGDTOXNLo5hFpDyR6UJKlIBpQkqUgGlCSpSAaUJKlIBpQkqUgGlCSpSA4z16DcdNezTbWbtW9XmyuRNNLYg5IkFcmAkiQVyYCSJBXJgJIkFcmAkiQVyYCSJBXJYeaS2q6vq6BLvbEHJUkqkgElSSqSASVJKpIBJUkqkgElSSqSASVJKpLDzCW1hEPJ1Wr2oCRJRTKgJElFMqAkSUXqN6AiYueIuD4i7omIuyLir+vpW0fEtRFxX/1zXPvLlSSNFs30oJ4H3p+ZewMHAe+NiH2ADwHXZeYewHX1c0mSWqLfgMrMFZl5e/3774F7gJ2ANwMX1c0uAo5oV5GSpNFng4aZR8RkYH/gVmD7zFwBVYhFxHa9LDMPmAcwadKkwdQqSUVwSP3QaHqQRERsAXwdODUzn2x2ucxckJkzM3Pm+PHjB1KjJGkUaiqgImIsVTh9JTO/UU9+OCIm1PMnAI+0p0RJ0mjUzCi+AM4H7snMc7rN+iYwt/59LnB168uTJI1WzZyDejVwPHBnRCyqp50GnA1cFhEnAr8BjmlPiZKk0ajfgMrMHwDRy+zXtbYcSZIqXklCklQkA0qSVCQDSpJUJANKklQkA0qSVCQDSpJUJANKklQkA0qSVKQNupq5NJzcdNezTbWbtW9XmyuRNBD2oCRJRTKgJElFMqAkSUUyoCRJRTKgJElFMqAkSUVymLmK0uzQcEkjnz0oSVKRDChJUpEMKElSkQwoSVKRDChJUpEMKElSkQwoSVKRDChJUpEMKElSkQwoSVKR+g2oiLggIh6JiCXdps2PiOURsah+HNbeMiVJo00zPagLgTkNpp+bmdPrx3daW5YkabTrN6Ay8ybgd0NQiyRJLxrM1cxPjoi3AguB92fmY40aRcQ8YB7ApEmTBrE5qbOavdL6rH272lyJNDoMdJDE54DdgOnACuCTvTXMzAWZOTMzZ44fP36Am5MkjTYDCqjMfDgz12bmC8B5wIGtLUuSNNoNKKAiYkK3p0cCS3prK0nSQPR7DioivgYcAmwbEcuA04FDImI6kMBS4J1trFGSNAr1G1CZeVyDyee3oRZJkl7klSQkSUUazDBzFcQh0JJGGntQkqQiGVCSpCIZUJKkIhlQkqQiGVCSpCIZUJKkIjnMXKNes0P0JQ0te1CSpCIZUJKkIhlQkqQiGVCSpCIZUJKkIhlQkqQiOcxcDTn0enRasGBxp0sY0frbv/PmTRuiSoYHe1CSpCIZUJKkIhlQkqQiGVCSpCIZUJKkIhlQkqQiOcy8cK0e7u3w8fZrdh/P2rerzZVIw5s9KElSkQwoSVKRDChJUpH6DaiIuCAiHomIJd2mbR0R10bEffXPce0tU5I02jTTg7oQmNNj2oeA6zJzD+C6+rkkSS3Tb0Bl5k3A73pMfjNwUf37RcARLa5LkjTKDXSY+faZuQIgM1dExHa9NYyIecA8gEmTJg1wcxruHN4uaUO1fZBEZi7IzJmZOXP8+PHt3pwkaYQYaEA9HBETAOqfj7SuJEmSBh5Q3wTm1r/PBa5uTTmSJFWaGWb+NeBHwF4RsSwiTgTOBg6NiPuAQ+vnkiS1TL+DJDLzuF5mva7FtUiS9CKvJCFJKpJXM5c6pFNXPV+wYHFL1ye1iz0oSVKRDChJUpEMKElSkQwoSVKRDChJUpEMKElSkRxmLkmF6OsrAPPmTRvCSspgD0qSVCQDSpJUJANKklQkA0qSVCQDSpJUJANKklQkh5lLI0SzV0eXhgt7UJKkIhlQkqQiGVCSpCIZUJKkIhlQkqQiGVCSpCI5zLxDHBIsSX2zByVJKpIBJUkqkgElSSrSoM5BRcRS4PfAWuD5zJzZiqIkSWrFIInXZubKFqxHkqQXeYhPklSkwfagEvh+RCTwhcxc0LNBRMwD5gFMmjRpkJuTNFj3PrimqXZ77ji2zZVIfRtsD+rVmTkD+FPgvRExq2eDzFyQmTMzc+b48eMHuTlJ0mgxqIDKzAfrn48AVwIHtqIoSZIGHFARsXlEbLnud+ANwJJWFSZJGt0Gcw5qe+DKiFi3nq9m5jUtqUqSNOoNOKAy8wFgWgtrkSTpRQ4zlyQVyauZS4XzyvcarexBSZKKZEBJkopkQEmSimRASZKKZEBJkopkQEmSiuQw8xZzSLCkkWLBgsV9zp83r73XarAHJUkqkgElSSqSASVJKpIBJUkqkgElSSqSASVJKpLDzKUR6JuXe3Prkaa/Id99afdw8HaxByVJKpIBJUkqkgElSSqSASVJKpIBJUkqkgElSSrSsBxm3okrhs/at2vIt6nhr7/h3ocfM3XAy7bbvQ+u6ch299xxbEe2O5INZoh6J9mDkiQVyYCSJBXJgJIkFWlQARURcyLiFxFxf0R8qFVFSZI04ICKiDHAZ4A/BfYBjouIfVpVmCRpdBtMD+pA4P7MfCAznwMuAd7cmrIkSaNdZObAFow4GpiTme+onx8PvDIzT+7Rbh4wr366F/CLgZc7pLYFVna6iEEYzvVbe+cM5/qtvXMGW//KzJzTc+JgvgcVDaatl3aZuQBYMIjtdERELMzMmZ2uY6CGc/3W3jnDuX5r75x21T+YQ3zLgJ27PZ8IPDi4ciRJqgwmoG4D9oiIKRGxCXAs8M3WlCVJGu0GfIgvM5+PiJOB7wFjgAsy866WVdZ5w+6wZA/DuX5r75zhXL+1d05b6h/wIAlJktrJK0lIkopkQEmSimRANRARp9SXcLorIv6lnjY5Ip6JiEX14/OdrrORRrXX0z9cX5LqFxHxxk7W2JuImB8Ry7vt48Pq6cXv+95qr+cVv+8BIuIDEZERsW39/JCIeKLba/qHTtfYlwb1R0T8a73v74iIGZ2usaeIOLOubVFEfD8idqynF7/v+6i9dfs9M310ewCvBf4T2LR+vl39czKwpNP1DbD2fYDFwKbAFOCXwJhO19ug/vnABxpMHw77vrfah8u+35lqwNOvgW3raYcA3+50bYOo/zDgu1Tf2TwIuLXTdTao+2Xdfn8f8Pnhsu/7qL1l+90e1PreDZydmasBMvORDtezIXqr/c3AJZm5OjN/BdxPdakqtd9w2ffnAn9Hgy/bDxON6n8zcHFWfgxsFRETOlJdLzLzyW5PN2cY7f8+am/Zfjeg1rcncHBE3BoRN0bEK7rNmxIRP6unH9ypAvvQW+07Ab/t1m5ZPa1EJ9eHBS6IiHHdppe+76Fx7cXv+4g4HFiemY1uu/qqiFgcEd+NiH2HurZm9FF/8fseICL+MSJ+C/wl0P1Q3nDY941qb9l+H5a3fB+siPhPYIcGsz5CtU/GUXVNXwFcFhG7AiuASZm5KiIOAK6KiH17fIpouwHW3tRlqYZCP/V/DjiTqrYzgU8Cb2d47Pveai9i3/dT+2nAGxrMux3YJTOfqs+pXQXs0b4qezfA+ovf95l5dWZ+BPhIRHwYOBk4nUL2/QBrb91+7/RxzNIewDXAId2e/xIY36DdDcDMTtfbTO3Ah4EPd5v+PeBVna63n9cymV7OO5W473urvfR9D+wHPAIsrR/PA78BdmjQdin1+Z1SHn3VD3wBOK5b218AEzpdcx+vZZc+3vPF7fveam/lfvcQ3/quAmYDRMSewCbAyogYH9U9sKh7JXsAD3SsysYa1k51CapjI2LTiJhCVftPOlZlL3ocpz4SWFJPL37f91Y7he/7zLwzM7fLzMmZOZnqcMyMzHwoInaIiACIiAOpTgms6mC56+mrfqp9/9Z6VNlBwBOZuaKT9fYUEd17RYcDP6+nF7/ve6udFu73UXmIrx8XABdExBLgOWBuZmZEzALOiIjngbXAuzLzd50stIGGtQN3RcRlwN1UnzDfm5lrO1hnb/4lIqZTHQ5YCryznj4c9n3D2jNzuOz7Ro4G3l3v92eAY+v303DxHaoRZfcDTwNv62w5DZ0dEXsBL1CNQHxXPX047Pveam/ZfvdSR5KkInmIT5JUJANKklQkA0qSVCQDSpJUJANKklQkA0qSVCQDShqkiDgpIh6IiOcj4gs95o2LiIcjYrcWbOeKiPjbwa5HGi78HpQ0CBHxR1RXjTgG+DHw+8x8qtv8j1NdombQXxKNiP2AG4EpmfnEYNcnlc4elDQ4h1Ndg+zKzFzRI5z+AHgHcH4rNpSZd1Jd4umvWrE+qXQGlDRAEXEv8M/AtKju5HpljyaHUV0G5ofdljkmIlZHxC7dpn06In4ZEds3sdlvAse1oHypeAaUNHB/AtwL/F9gAjC3x/yDgZ/2uIbaFcCd9TJExAeoAmdOZj7cxDZ/AhwYEZsNsnapeF4sVhq4J4FdgR/WV8/uaReqe1m9qL7w8GnAf0TEL6nuZzQ7M+9rcpsPAmOBHalupyKNWPagpIGbSvUhb1Ev8zcDnu05MTO/D9wGfAz488y8bQO2+Uy3dUsjmgElDdx04NeZ+Xgv81dS3eH4JVSiVEUAAAFKSURBVCJiNjCN6s6j6x3Wi4hvRcQZEfGjiFgWETO6zd66/vno4EqXymdASQM3nd57TwA/A/bpPiEipgHfAE6husHkWQ2Wmwosz8xXAf9EdQPE7vMebPJ8lTSsGVDSwPUXUN8D9o6IbQDqkXvfAc7JzAuA04FDI+KQdQtExJbARpm57gu/Y3jpnVQPBq5p2SuQCmZASQNQ34775fQRUPX3ln5Cdcv3ramC5duZeUY9fwlwOS/tRU2lOj+1zn7Ut4+PiC6q3tR5rXslUrm8koTURhExB/g0sE8zt3qPiJOAHTLzzPr5j4E3Z+bDEfHe+vc3tLVoqRD2oKQ2ysxrgM8AE5tcZD/gDnixl7Zdt/NNa6jOXUmjgj0oSVKR7EFJkopkQEmSimRASZKKZEBJkopkQEmSimRASZKKZEBJkopkQEmSimRASZKK9P8BZNKZmUJCMR8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 504x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(figsize = (7,5))\n",
    "sns.distplot(f0, bins = 25, kde = False, \n",
    "             color = 'cornflowerblue', label = 'Class 0')\n",
    "sns.distplot(f1, bins = 25, kde = False, \n",
    "             color = 'darkblue', label = 'Class 1')\n",
    "ax.set_xlabel(r\"$f\\hspace{.25}(x_n)$\", size = 14)\n",
    "ax.set_title(r\"Histogram of $f\\hspace{.25}(x_n)$ by Class\", size = 16)\n",
    "ax.legend()\n",
    "sns.despine()"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
